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Summary 

This  paper  deals  with  an  investor  loss  function  for  earnings 
forecasts.   Specifically  we.  develop  a  theoretical  framework  that 
measures  loss  in  the  context  of  the  problem  of  resource  allocation 
under  uncertainty.   The  framework  maps  a  set  of  forecasts  into  an 
expected  return.   This  expected  return  is  compared  to  that  return 
which  could  be  expected  given  perfect  knowledge  of  future  earnings 
and  the  resulting  difference  measures  the  investor's  loss.   Some 
empirical  results  are  given. 


This  paper  deals  with  an  investor  loss  function  for  earnings  fore- 
casts.  Specifically  we  develop  a  th  oretical  frar.ework  that  measures  loss 
in  the  context  of  the  problem  of  resource  allocation  under  uncertainty. 
The  framework  naps  a  set  of  forecasts  into  an  expected  return.   This  ex- 
pected return  is  compared  to  that  return  which  could  be  expected  given 
perfect  knowledge  of  future  earnings  and  the  resulting  difference  mea- 
sures the  investor's  loss. 

The  need  for  a  loss  function  arises  from  the  need  of  investors  to 
assess  the  value  of  alternative  forecasts.   In  addition  empirical  re- 
search studies  have  often  compared  various  forecast  models.   The  typical 
approach  has  been  to  compare  forecast  accuracy  or  dispersion.   This 
approach,  however,  is  limited.   For  example  Foster  [1977,  p.  10]  stated 
"it  is  important  to  recognize  that  our  measures  of  dispersion  are  essen- 
tially surrogate  criteria  for  evaluating  alternative  forecasting  models. 
A  more  complete  analysis  would  specify  the  loss  function  in  a  specific 
decision  context."   In  addition  Gonedes  et  al  [1976,  p.  94]  vrote,  "There 
is  a  more  fundamental  deficiency  in  these  prediction  performance  studies... 
Specifically  they  are  not  based  upon  any  explicit  theoretical  structure 
that  connects  their  frameworks  to  resource  allocation  under  uncertainty." 

The  purpose  of  the  present  study  is  to  develop  a  theoretical  frame- 
work to  overcome  these  limitations.   This  framework  takes  the  form  of  an 
investor  loss  function  (henceforth  ILF) .   A  secondary  purpose  is  to  em- 
pirically apply  the  ELF  for  purposes  of  comparing  forecasts  generated 
from  several  models  that  have  recently  been  employed  in  the  literature. 

The  paper  is  in  four  parts.   Part  one  discusses  some  of  the  issues 
associated  with  an  ILF,  part  two  presents  an  operational  form  of  the 
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ILF,  part  three  is  an  empirical  application  and  part  four  contains  sum- 
mary and  conclusions. 

BACKGROUND  ISSUES 
An  important  result  of  research  on  the  information  content  of  ac- 
counting earnings  is  that  ex  ante  knowledge  of  annual  earnings  can  provide 
an  opportunity  for  an  investor  to  earn  an  abnormal  return.   For  example, 
Ball  and  Brown  [1968]  reported  that  if  one  were  to  know  the  sign  of  the 
unexpected  annual  earnings  change  12  months  in  advance  it  would  be  possible 
to  earn  a  7%  abnormal  return  via  a  simple  trading  rule.    These  findings 
pose  some  interesting  questions  with  respect  to  earnings  forecasts. 

1)  If  the  market  is  efficient,  then  we  wouldn't  expect  a  forecast 
model  to  enable  an  individual  to  earn  an  abnormal  return  via 
utilizing  only  publicly  available  information. 

2)  If  the  expectation  in  (1)  is  correct,  then  from  the  individual 
investor's  standpoint  the  problem  of  comparing  various  sets  of 
forecasts  might  become  irrelevant.   One  could  expect  to  earn 
no  better  than  the  risk  conditioned  rate  of  return  since  the 
risk-return  relationship  should  not  depend  on  the  portfolio 
selection  process. 

3)  Looking  at  (2)  from  another  side,  we  would  not  expect  a  return 
less  than  the  risk  conditioned  return  via  the  same  reasoning. 

Given  the  above  reasoning,  it  might  seem  that  investors  would  only 
be  concerned  with  constructing  minimum  variance  portfolios  and  would  not 
be  concerned  with  using  earnings  forecasts  in  their  decision  models.   Re- 
search, however,  points  to  the  exact  opposite.   For  example,  Nordby  [1973] 
found  that  99%  of  responding  analysts  claimed  that  ':hey  use  earnings 
forecasts  in  their  decision  making  process. 

What  then  accounts  for  the  extensive  use  of  forecasts  in  practice? 
The  answer  seemingly  must  be  one  of  two  things: 
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(1)  Some  individuals  do  earn  an  abnormal  return  through  use  of 
forecasts  and  the  market  is  not  efficient. 

(2)  Some  individuals  think  that  they  can  earn  an  abnormal  return, 
but  the  market  is  efficient  and  they  are  possibly  irrationally 
allocating  resources  for  purposes  of  obtaining  forecasts. 

Research  and  reasoning  can  be  used  to  support  both  (1)  and  (2).   A 
large  amount  of  research  has  been  conducted  which  favors  efficiency.   On 
the  other  hand  there  is  evidence  which  is  not  consistent  with  efficiency. 
A  good  example  of  this  is  the  Value  Line  Investment  survey.   Black  [1973] 
found  reasonably  strong  evidence  that  the  survey  is  able  to  predict  returns 
of  securities  in  a  way  that  cannot  be  accounted  for  by  differences  in  risk. 
(Note  that  the  survey  relies  heavily  on  a  determination  of  "earnings 
momentum".)   In  addition,  Joy  et  al  [1977,  p.  207]  presented  evidence  that, 
"...the  information  contained  in  quarterly  earnings  was  not  fully  impounded 
into  stock  prices  at  the  time  of  announcement." 

It  is  not  the  purpose  of  the  present  paper  to  take  a  position  on  effi- 
ciency or  lack  of  efficiency  in  the  market.   However  it  must  be  noted 
that  the  theory  of  market  efficiency  does  not  explain  the  empirically 

observed  behavior  of  users  of  earnings  forecasts.   An  alternative  frsme- 

2 

work  which  might  explain  such  behavior  is  that  of  Grossman  and  Stiglitz 

[1976]  who  pointed  out  that  costless  information  is  not  only  sufficient 
for  market  efficiency  but  necessary  as  well.  Their  alternative  is  sum- 
marized (p.  218): 

In  the  structure  we  have  developed,  the  market  never 
fully  adjusts.   Prices  never  fully  reflect  all  the 
information  possessed  by  the  informed  individuals. 
Capital  markets  are  not  efficient,  but  the  difference 
is  just  enough  to  provide  the  revenue  required  to  com- 
pensate the  informed  for  purchasing  the  information. 

Note  that  this  framework  allows  for  the  possibility  that  there  is  a  value 

of  gathering  information  and  therefore  a  corresponding  loss  for  gathering 
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information  which  is  less  than  optimal.   (It  is  this  loss  that  is  the 
focus  of  the  present  study.)   We  define  optimal  information  to  be  that 
information  which  produces  the  maximum  possible  net  revenue  associated 
with  its  use,  where  net  revenue  is  computed  as  the  difference  between 
the  revenue  associated  with  the  use  of  the  information  and  the  acquisi- 
tion cost  of  that  information. 

If  the  market  is  assumed  to  be  efficient  and  the  market's  earnings 
expectation  model  is  assumed  to  be  optimal,  then  one  might  gauge  the  use- 
fulness of  a  given  forecast  method  by  measuring  the  abnormal  returns 
associated  with  an  investment  strategy  based  on  an  ex  ante  knowledge  of 
the  forecast  error  (i.e.,  unexpected  earnings)  of  the  given  model.   This 
statement  is  elaborated  on  by  Brown  and  Kennelly  [1972,  p.  404]: 

This  experimental  design  permits  a  direct  comparison 
between  alternative  forecasting  rules. .. .The. . .conten- 
tion is  based  on  the  hypothesis  (and  evidence)  that  the 
stock  market  is  "both  efficient  and  unbiased  in  that, 
if  information  is  useful  in  forming  capital  asset 
prices,  then  the  market  will  adjust  asset  prices  to 
that  information  quickly  and  without  leaving  any 
opportunity  for  further  abnormal  gain"  (Ball  and  Brown, 
1968).   There  is,  then  a  presumption  that  the  con- 
sensus of  the  market  reflects,  at  any  point,  an  esti- 
mate of  future  EPS  which  is  the  best  possible  from 
generally  available  data.   Since  the  abnormal  rate 
of  return  measures  the  extent  to  which  the  market  has 
reacted  to  errors  in  its  previous  expectations,  the 
abnormal  rate  of  return  can  be  used  to  assess  the 
predictive  accuracy  of  any  device  which  attempts  to 
forecast  a  number  that  is  relevant  to  investors.   To 
our  knowledge,  Ball  and  Brown  (1968)  were  the  first 
to  make  use  of  this  fact. 

This  basic  type  of  reasoning  can  be  used  to  derive  several  types 

of  empirical  tests  all  based  on  different  sets  of  assumptions  as  listed 

below. 
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Category       Assumptions  Test 

1  A)   Efficient  rcarket         Compare  prediction  models 
B)   Earnings  has  infor-      on  ability  to  approximate 

nation  content  market  prediction  model 

(assumed  to  be  optimal  as 
defined  above) 

2  A)   Efficient  market 

B)   Given  prediction        Information  content 
model  approximates 
market  model 

3  A)   Efficient  market         Joint  test  of 

information  content 
and  prediction  model 


Most  of  the  accounting  research  has  fallen  in  one  of  the  above 
three  categories.   For  example  Fester's  [1977]  study  on  quarterly 
accounting  data  falls  into  category  1  while  Ball  and  Brown  [1968]  and 
Brown  and  Kennelly  [1972]  among  many  others  fall  into  category  2.   It 
can  be  argued  also  that  categories  1  and  2  are  subsumed  under  category  3 
which  reduces  to  the  first  two  cases  if  the  assumptions  are  correct. 
In  the  present  study  we  develop  a  methodology  (ILF)  for  comparing  pre- 
diction methods  which  dees  not  rely  on  any  of  the  Table  1  assumptions. 
Instead  we  assume  that  there  exists  some  market  expectation  (prediction) 
for  earnings  (not  necessarily  optimal  as  defined  above)  which  has  been 
impounded  in  the  market  equilibrium  (price).   The  error  in  this  market 
earnings  expectation  has  not  been  impounded  by  the  market.  An  investor 
who  had  knowledge  of  this  (ncn- impounded)  information  would  be  able  to 
use  it  to  formulate  an  investment  strategy  which  would  produce  an  abnor- 
mal return  (as  measured  by  the  market  model).   Therefore  we  define  the 
investor's  loss  (due  to  inaccurate  earnings  predictions)  to  be  the 
difference  between  the  abnormal  return  he/she  could  earn  given  a 
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strategy  based  on  correct  forecasts  of  earnings  Cand  thereby  of  the 
non- impounded  information)  and  the  return  he/she  could  earn  given  a 
strategy  based  on  less  than  perfect  forecasts.   This  definition  is 
operationalized  below  and  discussed  in  the  context  of  decision  theory. 
It  is  also  applied  to  a  comparison  of  several  forecasts  models  found 
in  the  literature. 

It  should  be  emphasized  that  in  developing  the  loss  function  we  are 
not  concerned  with  market  efficiency  per  se  but  rather  an  individual  in- 
vestor's perceptions  with  respect  to  market  efficiency.   In  particular  we 
assume  that  the  investor  believes  that  there  is  a  possibility  of  earning 
a  return  higher  than  predicted  by  the  Sharp-Lintner  [1964,  1965]  capital 
asset  pricing  model.  As  pointed  out  previously,  unless  this  assumption 
holds,  there  is  no  private  value  of  earnings  forecasts  bcsed  on  publicly 
available  information.   If  there  is  no  private  value,  then  from  the  in- 
dividual's standpoint  the  process  of  comparing  forecasts  is  dubious. 
It  is  also  pointed  out  that  the  loss  function  derived  in  this  paper  does 
not  depend  on  the  need  to  compare  forecast  methods  but  simply  specifies 
the  loss  associated  with  different  forecast  sources.   If  all  forecast 
methods  have  equal  loss,  then  there  is  no  need  to  compare  forecast 
methods.   This  is  an  empirical  question. 

CFERATIONALIZATION  OF  THE  ILF 
In  order  to  use  the  ILF,  it  is  first  necessary  to  operationally  define 
a  market  earnings  expectation  model.   In  this  study  we  use  the  cross-sectional 
model  employed  by  Ball  and  Brown  [1968]  which  regresses  individual  firms' 
earnings  changes  on  market  earnings  changes.   I'e  use  this  model  since  Ball 
and  Brown  observed  that  ex  ante  knowledge  of  its  residuals  made  it  possible 
for  an  investor  to  earn  a  7%  abnormal  return. 
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To  facilitate  operationalization  of  the  loss  function  we  make  the 

following  definitions: 

(1)     E(F        |r  ) 
0 

(2)  a  -  ;±  +  ;±  z  a  +• 

j=i 

N 

(3)  E(F        |*   )   =  a     +  b      (  E      E(F        |r  )  *     =  {A       },   j#L 

xc0  x  j-1         J   0  J   0 

(4)  E[ln(l  +  R       -   R     )]    =   B     E(ln(l  +  R       -  R     )) 

it  it  l  mt  rt 


Where: 


a)  (1)  represents  the  investor's  expectation  of  earnings  change 

for  firm  i  and  period  t~.  This  expectation  could  be  the  re- 
sult of  intuition,  statistical  modeling  or  judgmental  opinion 
and  is  conditioned  upon  r,  the  set  of  prior  earnings  for  firm  i. 

b)  (2)  is  an  empirical  description  of  the  relationship  between 
market  earnings  changes  and  firm  i  earnings  changes,  A.  .   The 

coefficients  a.  and  b  are  assumed  to  be  estimated  by  the  investor. 

Note  that  the  investor  is  assumed  to  use  the  same  coefficients 


in  (3) .   Also  the  subscript  t  denotes  time  previous  to  t 


:■' 


(In  the  empirical  application,  a.  and  b   are  estimated  by 
regression  on  previous  years'  data.) 

c)  (3)  represents  the  investor's  expectation  of  earnings  changes 

(A   )  for  firm  i  in  period  t.  conditioned  upon  his  expectation 
itQ  U 

of  the  market  earnings  ($  )  in  period  tn.   Since  this  depends 

on  E(F.   |r. )  (which  is  ex  ante),  it  is  ex  ante.  The  coeffi- 

Ja0   1   a 
cients  a.  and  b.  are  from  (2)  and  are  taken  as  known  in  the 
equation. 

d)  (4)  is  the  log  form  Sharp-Lintner  [1964,  1965]  capital  asset 

pricing  nodel  where  R.   represents  the  return  on  asset  i  in 

period  t,  R   represents  the  market  return  in  period  t  and  Rc 
mt  t 

is  the  risk  free  rate  of  return. 
Given  the  above  definitions  we  can  define  the  investor's  anticipation 
of  the  individual  components  of  change  in  earnings.   It  is  this  individual 
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component  which  Ball  and  Brown   [1968]    found  to  enable   one   to  earn  an 
abnormal  return  when  known  in  advance.      We  proceed   to   define  the   in- 
vestor's  anticipation  of  the  individual  component   of  changes  in  earnings 
subtracting  (3)    from   (1): 

(5)   ait  '  E(Fit  'V  -  E(Fit  'V 

0        0  0 

When  a    is  greater  than  zero,  then  the  investor  expects  a  positive 
0 
individual  component  of  change  in  earnings.  When  a.   is  less  than  zero, 

ato 

a  negative  individual  component  of  change  is  anticipated.   This  is  con- 
sistent with  the  Ball  and  Erown  market  conditioned  definition  of  indi- 
vidual component  of  change  in  earnings  e>xept  it  is  based  en  predicted 
earnings  as  opposed  to  actual  earnings. 

Given  that  a.   is  positive  (negative),  the  investor  would  be  expected 
to  buy  long  (sell  short)  in  asset  i.   Also  to  the  degree  that  his  expectations 
are  correct,  he  will  earn  an  abnormal  return,  AEC,.   Similarly  let  AR^  represent 
the  abnormal  return  assuming  that  the  investor  has  perfect  forecasts  of 
the  future  earnings  (i.e.,  his  predictions  of  all  firms  are  perfectly 
accurate).  Then  define  the  investors  loss  function  (ILF)  as 


(6)   ILF  =  ARpp  -  ARp 


where  ILF  has  the  interpretation  as  being  the  loss  incurred  from  not 
having  perfect  forecasts.   The  minimum  expected  value  is  expected  to  be 
0  in  the  case  of  having  perfect  forecasts,  and  equal  to  the  abnormal 
return  of  having  perfect  forecasts  in  the  case  of  having  useless  fore- 
casts. 
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A  Further  Interpretation 

In  terms  of  standard  decision  theory  the  ILF  measures  the  cost,  in 
terms  of  return,  of  not  having  perfect  forecasts.   A  larger  abnormal 
return  earned  means  a  smaller  loss.   This  relationship  is  depicted  in 
Figure  1.   In  point  A  the  loss  is  at  a  maximum  and  the  abnormal  return 

[Figure  1  about  here] 
is  at  a  minimum.   Point  B  depicts  the  case  of  perfect  forecasts. 

Note  that  the  loss  depends  strictly  on  the  abnormal  return.   This 
is  important  because  abnormal  returns  are  risk  adjusted  or  independent 
of  the  market.   This  means  that  forecasting  methods  can  be  preference 
ranked  based  on  the  ILF  without  a  separate  consideration  of  risk. 

EMPIRICAL  APPLICATION 
Forecast  >!odels 

The  empirical  results  of  this  study  focus  on  the  ability  of  several 
statistical  models  to  predict  annual  EPS  from  quarterly  EPS.   This  purpose 
has  been  suggested  by  the  Financial  Accounting  Standards  Board  in  the 
discussion  memorandum,  Interim  Financial  Accounting  and  Reporting  (.FASB, 
1978).   In  addition  there  has  been  a  considerable  amount  of  research 
done  on  the  predictive  ability  of  models  using  quarterly  EPS  (e.g., 
Lcrek,  1979;  Foster,  1977;  Brown  and  Rozeff,  1978). 

We  focus  on  several  models  that  have  been  given  considerable  atten- 
tion with  respect  to  their  ability  to  represent  the  time  series  of 
quarterly  EPS.   These  are 

1)    a  seasonally  and  consecutively  differenced  first  order  and 
seasonal  moving  average  model  [Griffin,  1977;  Watts,  1975] 
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Figure  1 
The  Relationship  Between  Loss  and  Abnormal  Return 


Loss 


Abnormal  Return 
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2)  a  seasonally  differenced  first  order  autoregression  model 

3 

with  a  constant  drift  term  [Fester,  1977] 

3)  a  seasonally  differenced  first  order  autoregressive  and 
seasonal  coving  average  model  [Brown  and  Rozeff,  1978] 

4)  individually  identified  and  estimated  Box-Jenkins  models 
for  each  firm. 

la  the  Bos-Jenkins  notation  the  first  three  are  referred  to  as 

(0,1,1)  x  (0,1,1),  (1,0,0)  x  (0,1,0)  and  (1,0,0)  x  (0,1,1),  respectively. 

For  the  remainder  of  the  study  these  are  referred  to  as  the  GW,  F,  and 
BR  model's. 

Population  Studied 

Data  pertaining  to  267  firms  was  obtained  from  the  Ccmpustat  quar- 
terly and  CRSP  monthly  tapes.   For  a  firm  to  be  included  in  the  group, 
it  was  required  to  have  no  missing  EPS  or  returns  data  for  the  64  con- 
secutive quarters  beginning  with  the  first  quarter  of  1962.   This  pro- 
vided a  sample  period  frcm  1962  through  1977.   The  EPS  number  used  was 
primary  earnings  per  share  excluding  extraordinary  items  and  discontin- 
ued operations,  adjusted  for  capital  changes.   The  return  figure 
selected  from  CRSP  included  both  a  dividend  and  price  component. 

Note  that,  unlike  previous  research,  all  firms  which  met  the  sur- 
vivorship test  were  retained  for  analysis.   We  define  this  group  to  be 
the  population  of  interest  and  make  no  attempt  to  generalize  to  a  larger 
number  of  years  or  group  of  firms.   To  use  statistical  testing  to  make 
inferences  about  a  larger  group  of  firms  would  be  unwarranted  because 
there  is  no  reason  to  believe  that  firms  which  fail  to  meet  the  survivor- 
ship test  are  the  same  as  those  that  do.   In  fact,  a  priori  reasoning 
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indicates  that  firns  meeting  the  test  are  very  likely  to  be  larger  and 
less  risky  than  on  the  average.  Also  attempting  to  generalize  across 
all  years  would  be  unwarranted  because  structural  changes  in  the  economy 
might  produce  a  shifting  in  the  relative  performance  of  different  fore- 
cast methods.  Even  if  this  was  not  a  problem,  in  order  to  generalize 
to  all  years,  it  would  be  necessary  to  obtain  a  reasonably  large  random 
sample  of  years.  This  is  not  possible  because  of  limited  data  avail- 
ability. 

Since  statistical  testing  is  used  for  making  inferences  about  a 
larger  population  and  under  the  circumstances  we  felt  that  such  infer- 
ences would  be  unwarranted,  no  statistical  tests  are  presented  in  this 
paper.  Instead  our  goal  is  to  present  results  for  an  entire  population 
therefore  avoiding  the  need  to  make  statistical  generalizations.   We  feel 
that  this  approach  is  useful  because  it  presents  results  for  a  large 
population  which  is  of  interest  in  its  own  right. 


AcDlicaticn  of  the  Forecasting  Models 


For  purposes  of  assessing  the  ILF's  of  the  4  forecast  methods,  the 
years  of  1974  through  1977  wera  used  as  holdout  periods.   Therefore  the 
267  series  were  each  modeled  16  times,  once  for  each  method  using  pre 
1974  data  (48  quarters  in  the  base  period)  and  again  for  each  method 
(52  quarters  in  the  base  period)  using  all  pre  1975  data,  etc.  The  re- 
sult was  that  each  model  made  predictions  for  the  4  quarters  in  each  of 
the  four  hold  out  years.   These  quarterly  forecasts  were  aggregated 
within  each  year  to  form  annual  forecasts.   These  forecasts  represent 

F    in  (1)  above. 
Z0 
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Next  the  coefficients  a,  and  b.  in  (3)  were  estimated  for  each 

i      i 

firm.   The  procedure  was  done  for  each  hold  out  year  and  was  based  on 
all  data  prior  to  the  holdout  year.   The  market  index  was  a  weighted 
average  of  all  of  the  sample  firms'  EPS  except  the  one  for  which  the 
model  was  being  estimated.   The  residuals  of  the  models  were  tested  for 
autocorrelation  and  the  null  hypothesis  of  no  autocorrelation  was  re- 
jected for  only  8  firms  which  was  attributed  to  chance. 

The  a  and  b.  coefficients  were  then  applied  to  compute  the  in- 
vestor's anticipation  of  the  market  conditioned  EPS  in  (3)  and  finally 
the  anticipation  of  the  unexpected  earnings  change  in  (5). 

Application  of  the  Market  Model 

The  equilibrium  market  model  (4)  was  estimated  for  each  firm  and 
for  each  of  the  four  years.   The  estimation  included  data  in  the  5  years 
preceeding  the  holdout  year.    The  residuals  from  these  models  when 
applied  to  the  four  holdout  years  constitute  abnormal  returns.   The  mar- 
ket index  used  was  the  value  weighted  market  index  containing  the 
dividend-price  returns  of  all  firms  as  supplied  on  the  CRSP  tape. 

Empirical  Results:  Losses 

The  loss  for  each  forecast  method  was  calculated  by  computing  the 

annual  cumulative  abnormal  return  (CAR)  associated  with  each  forecast 

method  and  subtracting  this  from  the  CAR  associated  with  an  investment 

strategy  based  on  ex  ante  knowledge  of  the  actual  EPS.   The  CARs  were 

computed  by  assuming  a  long  investment  given  a.    in  (5)  was  positive 

"o 
(henceforth  CAR+)  and  a  short  investment  given  that  a    was  negative 

1C0 
(henceforth  CAR-) . 
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Table  1  gives  the  cumulative  abnormal  residuals  for  the  4  forecast 
methods  and  actual  EPS,  for  the  12  months  prior  to  and  including  the 
earnings  announcement  date.   For  example,  long  investments  for  the  BR 
method  made  11  months  prior  to  the  annual  announcement  month  and  termi- 
nating at  the  end  of  the  announcement  month  show  an  abnormal  return  of 
.00391  for  1976.   Quick  inspection  reveals  that  only  the  GW  (for  the 
year  1977)  and  the  actuals  demonstrate  a  strong  apparent  pattern  of 
abnormal  return. 

Table  2  presents  the  loss  (as  defined  in  equation  6)  for  each  of 
the  four  methods.  Note  that  in  all  cases  the  loss  is  positive  with  the 
smallest  average  loss  being  associated  with  the  GW  method  and  the  larg- 
est average  loss  being  associated  with  the  BJ  method.   Also  note  that 
the  GW  method  had  the  smallest  loss  in  three  out  of  the  four  years  with 
1975  being  the  exception.   In  addition,  the  percentage  difference  be- 
tween the  GW  and  ether  methods  was  substantial  for  1976  and  1977.   This 
can  be  seen  by  examining  the  ratio  of  the  next  smaller  loss  to  the  GW 
for  these  two  years.   This  ratio  is  1.39  (.0501  *  .0360)  for  1976  and 
1.99  (.0730  *  .0367)  for  1977.   Also,  with  the  exception  of  1975,  the 
percentage  differences  between  the  F,  ER  and  BJ  methods  were  small. 
Finally,  note  that  no  consistent  pattern  of  rankings  exists  between  these 
three  methods. 

Empirical  Results:   Forecast  Errors 

The  question  is  now  examined  as  to  whether  forecast  error  measures 
are  consistent  with  those  of  the  ILF.   Therefore  Table  3  presents  the 
average  absolute  percentage  forecast  errors  for  the  four  methods  over 
the  hold  out  period.   The  results  are  presented  for  the  quarterly  and 
annual  forecasts. 
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[Table  3  about  here] 

For  both  the  quarterly  and  annual  forecasts  the  rankings  based  on 
the  five  year  averages  are  fairly  consistent.   In  all  cases  the  BR  method 
has  the  smallest  error  and  the  F  method  has  the  largest  error.   Also  the 
BJ  method  performs  better  than  the  GW  in  all  cases  except  for  the  two 
quarter  ahead  forecasts.   These  results  are  consistent  with  those  of 
Collins  and  Kopwood  [1980]  who  found  identical  rankings  for  the  annual 
forecast  generated  prior  to  the  first  quarter  of  the  year. 

Table  4  presents  the  sane  error  analysis  as  Table  3  but  based  on 
the  mean  ranks.   This  information  is  presented  because  it  has  the  advan-" 
tage  of  not  depending  on  a  particular  choice  of  an  error  metric. 

[Table  4  about  here] 

Note  that,  while  the  average  quarterly  rankings  vary  depending  on 
the  forecast  horizon,  the  rankings  based  on  the  annual  error  are  identical 
to  those  based  on  the  absolute  percentage  error.   The  fact  that  the  rank- 
ings are  the  same  is  particularly  relevant  to  the  present  study  since  the 
losses  in  Table  2  are  based  on  annual  forecasts. 

Discussion  of  the  Empirical  Results 

One  thing  immediately  noticeable  about  the  results  is  that  the  ILF 
and  error  analysis  produced  different  rankings  of  the  methods.  The  ILF 
ranks  were:  GW=1,  F=4,  BR  =2  and  BJ  =  3,  and  the  error  analysis 
ranks  were:  GW  =  3,  F  =  4,  ER  =  1,  BJ  =  2.  This  discrepancy  can  be 
accounted  for  by  the  fact  that  for  a  given  investment  decision  the  size 
of  the  forecast  error  need  not  necessarily  be  related  to  whether  or  not 
the  best  Investment  decision  is  made.  For  example,  two  different  fore- 
casts can  be  quite  different  in  terms  of  accuracy  but  both  can  induce 
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the  same  investment  decision.   This  can  be  seen  from  the  situation  where 
the  magnitude  of  the  unexpected  earnings  implied  from  one  forecast  is 
much  larger  than  another  but  both  induce  a  buy  decision. 

The  net  result  is  that  the  investment  performance  is  determined  by 
the  percent  of  time  that  the  correct  decision  is  made  where  each  deci- 
sion outcome  is  weighted  by  its  investment  return  for  that  outcome. 
These  percentages  are  presented  in  Table  5.   Note  that  these  ranks  are 
consistent  with  those  of  the  ILF. 

[Table  5  about  here] 

Another  way  of  looking  at  investment  performance  is  by  considering 
the  percentage  of  time  that  a  given  method  results  in  the  same  decision 
which  would  have  been  made  had  the  forecast  been  perfectly  accurate. 
This  approach  will  be  exactly  the  same  as  that  presented  in  Table  5  if 
it  is  true  that  a  perfect  forecast  will  always  lead  to  the  correct  in- 
vestment decision.   One  reason  for  looking  at  investment  performance  in 
this  manner  is  that  it  is  that  it  eliminates  as  noise  the  cases  where 
utilization  of  a  perfect  forecast  leads  to  the  wrong  investment  decision. 
This  method  of  describing  investment  performance  will  henceforth  be 
referred  to  as  "weighted  agreement  with  the  actual"  (WAA)  and  the  first 
method  will  be  referred  to  as  "weighted  agreement  wTith  the  market"  (VAK) . 

Table  6  presents  the  WAA  results.   Notice  that  the  rankings  are 
again  the  same  but  the  means  are  larger.   Their  increased  magnitude  is 
expected  since  a  positive  abnormal  return  is  produced  any  time  that  the 
same  decision  is  made  as  would  be  made  with  a  perfect  forecast. 

[Table  6  about  here] 

Since  Tables  5  and  6  explain  the  losses  in  terms  of  the  combined 
percentage  of  investment  success  and  individual  decision  outcome  weighted 
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by  returns,  it  is  possible  to  further  explain  these  results  by  consider- 
ing how  much  of  each  percentage  is  due  to  the  percentage  of  success  and 
how  much  is  due  to  the  weighting.   To  this  end,  Table  7  presents  the 
percentage  of  time  that  each  forecast  method  led  to  the  same  decision 
as  would  have  been  made  had  the  forecast  been  perfect.   This  is  identi- 
cal to  WAA  but  the  weighting  has  been  eliminated.   Note  that  the  rank- 
ings are  not  the  same  as  in  WAA  in  the  case  of  the  F,  BR  and  BJ  methods. 
This  implies  that  making  the  correct  decision  more  often  does  not  neces- 
sarily imply  a  higher  average  return.   Keep  in  mind  however  that  is  for 
the  case  where  a  perfect  forecast  is  assumed  to  always  produce  the  cor- 
rect investment  decision. 

[Table  7  about  here] 

Table  8  removes  the  weighting  for  WAM.   The  rankings  are  fairly 
consistent  with  the  exception  of  a  tie  between  F  and  BR.   However,  this 
is  not  surprising  since  on  the  WAM  they  were  virtually  identical. 

[Table  8  about  here] 

In  summary,  the  conflict  in  rankings  between  the  ILF  and  error 
analysis  is  explained  by  the  WAM  and  AM  which  imply  that  the  method  with 
smaller  error  does  not  correspondingly  make  the  correct  investment  deci- 
sion a  larger  percent  of  the  time.   The  WAA  and  AA  imply  that,  given 
that  perfect  forecasts  always  lead  to  the  optimal  decision,  it  is  possi- 
ble for  a  method  to  fare  better  in  terms  of  the  number  of  times  that  it 
leads  to  the  correct  decision  but  fare  worse  in  terms  of  its  total  return. 

Summary  and  Conclusions 
Previous  research  involving  comparisons  among  forecast  methods  has 
typically  relied  on  various  error  metrics.   In  the  present  study  an 
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alternative  approach  has  been  taken,  namely  conparing  forecast  methods 
based  on  the  outcomes  of  investment  decisions  which  depend  on  earnings 
forecasts.   In  particular  an  investors  loss  was  defined  as:   the  dif- 
ference between  the  abnormal  return  he/she  could  earn  given  a  strategy 
based  on  correct  forecasts  of  earnings  (and  thereby  of  the  non-impounded 
information)  and  the  re'.urn  he/she  could  earn  given  a  strategy  based  on 
less  than  perfect  forecasts. 

Several  forecast  models  were  examined  based  on  their  observed  loss. 
The  results  indicated  that  the  models  studied  by  Griffin  and  Katts 
((0,1,1)  x  (0,1,1))  performed  better  than  those  of  Brown  and  Rozeff 
((1,0,0)  x  (0,1,1)),  Foster  ((1,0,0)  x  (0,1,0))  and  individually  identi- 
fied Box- Jenkins  models. 

Furthermore  these  same  models  were  examined  based  upon  an  outlier 
adjusted  mean  absolute  percentage  error  metric  and  a  mean  rank  criterion. 
These  rankings  were  found  to  be  identical  to  previous  research  on  a  dif- 
ferent sample  [Collins  and  Hopwood,  1980]  but  were  different  than  those 
produced  by  the  loss  function.   This  was  explained  by  showing  that  a 
smaller  forecast  error  did  not  lead  to  a  corresponding  increase  in  per- 
centage of  times  that  the  correct  investment  decision  was  made. 
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FOOTNOTES 

In  the  present  study,  we  define  unexpected  earnings  as  the 
unexpected  difference  betx;een  an  individual  firm's  earnings  and  the 
market  conditioned  expectation  of  the  same  number.   This  definition  is 
the  same  as  used  by  Ball  and  Brown  [1968], 

2 

The  reader  is  referred  to  the  Grossman  and  Stiglitz  [1976] 

paper  for  details  relating  to  the  assumptions  and  logic  of  their 
analysis. 

3 

In  the  present  study,  we  exclude  the  constant  term  based  on 

the  evidence  provided  by  Brown  and  Rozeff  [1978]  that  this  term  is  not 
significant. 

4^ 
For  1974,  there  were  only  4  years  of  data  available  for  re- 
gression estimation. 
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Table  2 

Losses  Associated  With  Each  of  the 
Four  Forecast  Methods* 


GW 

F 

BR 

BJ 

1974 

.0863 

.0938 

.0871 

.0925 

1975 

.0748 

.0473 

.0488 

.0701 

1976 

.0360 

.0597 

.0589 

.0501 

1977 

.0367 

.0774 

.0780 

.0730 

Mean 

.0585 

.0696 

.0682 

.0714 

Rank 

1 

3 

2 

4 

*Based  on  a  composite  investment. 


Table  3 

Mean  Absolute  Percentage  Forecast  Errors 
for  the  Four  Methods* 


1  quarter 
ahead 


2  quarters 
ahead 


Horizon 

3  quarters 
ahead 


4  quarters 
ahead 


Annual 


.3168049 
.3131468 
.3061375 
.3112299 


,3415264 
,3495478 
.3393090 
,3848678 


.3707929 
.3946708 
.3816767 
.3S65412 


.5296023 
.5308849 
.5381678 

.5247461 


.3030187 
.3069276 
.3082423 
.3225231 


.5466285 
.5173653 
.4795169 
.5150834 


.5507753 
.5714754 
,4954936 
,4876365 


.5682686 
.54963  54 
.4833038 
.4882317 


.6131159 
.5739701 
.5172408 
.5441138 


.5053421 
.4871677 
.4242318 
.4296102 


.3580204 
.4006113 
.3420058 
.3310875 


.3591631 
,4279471 
,3365913 
,3659078 


.3986770 
.4062940 
.3671510 

.3940491 


.5185392 
.5335978 
.4dl2312 
.4635779 


.3398120 
.3441070 
.3188220 
.3034558 


.3239681 
.3249985 
.3072657 
.3128735 

.3863555 
.3890305 
.3589815 
.3675686 


.3166372 
.3067774 
.3228528 
,3333958 

,3920255 
,4139370 
,3735617 
,3929520 


.3378085 
.3605339 
.3222657 
.3465253 

.4188867 
,4277835 
.38S5993 
,4038368 


.4160805 
.4476179 
.4003121 
.4405104 

.5193345 
.5215177 
.4842380 
.4932370 


.2949488 
.307a918 
.2880671 
.3082472 

.3607804 
.3615235 
.3348408 
.3409591 


*Errors  larger  than  3  were  set  equal  to  3, 


Table  4 

Mean  Ranks*  of  Forecast  Errors  Associated 
with  the  Four  Forecast  Methods 


Horizon 

1  quarter 

2  quarter 

3  quarters 

4  quarters 

ahead 

ahead 

ahead 
Year  1 

ahead 

Annual 

GW 

2.498127 

2.310861 

2.239700 

2.344569 

2.423221 

F 

2.513109 

2.438202 

2.576779 

2.543071 

2.438202 

BR 

2.329588 

2.397004 

2.584270 

2.535581 

2.468165 

BJ 

2.659176 

2.853933 

2.599251 
Year  2 

2.576779 

2.670412 

GW 

2.647940 

2.524345 

2.704120 

2.625468 

2.726592 

F 

2.535581 

2.749064 

2.576779 

2.610487 

2.655431 

BR 

2.370787 

2.397004 

2.367041 

2.292135 

2.314607 

BJ 

2.445693 

2.329588 

2.352060 
Year  3 

2.471910 

2.314607 

GW 

2.400749 

2.325S43 

2.479401 

2.438202 

2.468165 

F 

2.831461 

2.831461 

2.632959 

2.692884 

2.711610 

BR 

2.411985 

2.333333 

2.453184 

2.516854 

2.505618 

BJ 

2.355805 

2.509363 

2.434457 
Year  4 

2.352060 

2.314  607 

GW 

2.539326 

2.494382 

2.434457 

2.307116 

2.483146 

F 

2.408240 

2.340824 

2.475655 

2.565543 

2.337079 

BR 

2.543071 

2.486891 

2.554307 

2.441948 

2.479401 

BJ 

2.509363 

2.677903 

2.535581 
Average 

2.685393 

2.700375 

GW 

2.521536 

2.413858 

2.464419 

2.428839 

2.525281 

F 

2.572097 

2.589888 

2.565543 

2.602996 

2.535581 

BR 

2.413858 

2.403558 

2.489700 

2.446629 

2.439139 

BJ 

2.492509 

2.592697 

2.480337 

2.521536 

2.500000 

Table  5 

Weighted  Percentage  of  Times  that  the  Decision  Leading 
to  a  Positive  Return  Was  Made 


Overall 

Year  1 

Year  2 

Year  3 

Year  4 

Average 

Rank 

GW 

.4656 

.4270 

.4907 

.6603 

.5062 

1 

F 

.4445 

.5059 

.4097 

.5303 

.4712 

3 

BR 

.4633 

.5012 

.4179 

.5189 

.4746 

2 

BJ 

.4484 

.4416 

.4436 

.5453 

.4675 

4 

Table  6 

Weighted  Percentage  of  Tizies  that  the  Same  Decision  Was  Made 
as  Would  Have  Been  Made  Had  a  Perfect  Forecast  Been  Made 


Overall 

Year  1 

Year  2 

Year  3 

Year  4 

Average 

Rank 

GW 

.5380 

.5867 

.6781 

.6429 

.6080 

1 

F 

.2233 

.5944 

.6266 

.5188 

.5652 

3 

BR 

.5430 

.6138 

.6020 

.5607 

.5792 

2 

BJ 

.5054 

.5496 

.6422 

.5523 

.5603 

4 

Table  7 

Percentage  of  Times  That  the  Same  Decision  Was  Made  as  Would 
Have  Been  Had  the  Perfect  Forecast  Been  Made 


Overall 

Year  1 

Year  2 

Year  3 

Year  4 

Average 

Rank 

GW 

.5785 

.5709 

.6628 

.6207 

.6082 

1 

F 

.5517 

.5862 

.5939 

.5211 

.5632 

4 

BR 

.5900 

.6015 

.5824 

.5862 

.5900 

2 

BJ 

.5479 

.5709 

.6360 

.5824 

.5825 

3 

Table  8 

Percentage  of  Times  That  the  Decision  Leading  to 
a  Positive  Return  Was  Made 


Overall 

Year  1 

Year  2 

Year  3 

Year  4 

Average 

Rank 

GW 

.4904 

.4444 

.4789 

.6207 

.5086 

1 

F 

.4556 

.4751 

.4176 

.5211 

.4674 

3.5 

BR 

.4789 

.4904 

.3985 

.5019 

.4674 

3.5 

BJ 

.4751 

.4521 

.4598 

.5519 

.4847 

2 
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